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1. INTRODUCTION 


1.1 EXPERIMENT BACKGROUND 

The Foreign Commodity Production Forecasting (FCPF) Implementation Plan 
(ref. 1), dated January 15, 1980, provides for a category 3 (test and evalua- 
tion) experiment for U.S. /Canada wheat and barley to be completed In fiscal 
year 1980 (FY80). A wheat and barley labeling experiment plan (ref. 2) and a 
plan for an evaluation of a Procedure 1A technology (ref. 3) were developed to 
support that exploratory experiment. 

This document Is a detailed plan for a supplemental experiment to evaluate 
several crop growth stage and crop starter models. This experiment Is a joint 
effort of the Supporting Research (SR) and FCPF Projects of the Agriculture 
and Resources Inventory Surveys Through Aerospace Remote Sensing (AgRISTARS) 
Program. Ground observations of spring wheat and barley crop growth stages 
made during the 1979 crop year (CY79) In the U.S. Northern Great Plains 
(USNGP) and normal (historical) growth stage estimates will be compared with 
model predictions In these evaluations. These agro-meteorologlcal models were 
recommended for testing by the SR Project and are candidates for use during 
the U.S. /Canada wheat and barley pilot experiment In FY81. 

During the Large Area Crop Inventory Experiment (LACIE), spring wheat crop 
calendars based upon historical normals were Improved through the development 
of crop calendars adjusted by dally maximum and minimum temperatures and day 
lengths (ref. 4). Modifications to this basic model (ref. 5) have resulted In 
variations that offer potential for Improved accuracy In growth stage estima- 
tion. In addition, models for barley growth and planting data determination 
(ref. 6) have been Identified which may support development of Improved crop 
calendars. 

1.2 OBJECTIVES 

The general objectives of this experiment are to provide timely Information to 
aid In understanding crop calendars and to provide data that will support a 
selection between current crop calendar models. « 
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Recommendations on whether or not to use a specific combined planting date 
model and growth stage model (or models) In the follow-on exploratory and 
pilot experiments will be made after evaluating the results of this 
experiment. Recommendations for further research will also be developed. 

1.3 APPROACH 

ir 1 1 I 

The following crop growth models and starter models will be tested and 
evaluated In the experiment. 

a. Spring Wheat Growth Stage Models 

1. Robertson 

2. Improved Robertson, version i 

3. Improved Robertson, version 2 

b. Barley Growth Stage Model 
Williams 

c. Starter Model 
Feyerherm 

Comparisons of these models will be with normal (historical) growth stage 
estimates (Including planting dates) and with CY79 periodic observations made 
at 51 segments In the U.S. Great Plains (USGP). 

1.4 COMPONENTS' ROLES 

Component elements having major roles In the experiment are Crop Calendars, 
Experiment Design, and Accuracy Assessment. 

Component Implementation planning for this supplemental exploratory experiment 
Is described In subsequent sections followed by summaries of (a) data and 
» system requirements, (b) resources, and (c) an Integrated experiment schedule. 

Component responsibilities are stated In the FCPF Implementation Plan 
(ref. 1). Overall conformance to the technical aspects of this experiment 
will be monitored by a representative of the Experiment Design Component. 
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1.5 PRODUCTS. REPORTS. AND DOCUMENTATION 


Each of the technical components will Interchange and produce products as 
specified In the Individual component sections. Results from evaluations will 
be reported on a component basis to be compiled Into a Preliminary Technical 
Review Report (PTRR). Formal documentation of experiment results and recom- 
mendations for future pilot experiments and further development will follow 
the preliminary report. 
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2. EXPERIMENT DESIGN 


2.1 BACKGROUND 

In the LAC IE and presently In the FCPF Project, Inventories of specific crops 
of Interest are conducted over large areas using Lardsat satellite data. This 
requires the Identification of target samples by a labeling procedure for the 
crop of Interest. The spectral signature representing the target sample for a 
crop of Interest depends on a variety of factors such as crop development 
stage, soil type, moisture conditions, and haze. The most Important element 
for purposes of Identification of the crop type Is the crop development stage. 
Current labeling procedures depend upon the knowledge of what crop development 
stage Is expected for the crop of Interest during the times that the Landsat 
data were required. 

Beginning with LACIE, models have been used to predict crop development over 
extended areas. These models are referred to as agromet models because they 
estimate the dally rate of phenologlcal development as a function of day 
length and maximum and minimum air temperature. The model used during LACIE 
(ref. 4) was developed by Robertson for spring wheat. The basic form Is 

DID = [a^DL - a 0 ) + a 2 (DL - a 0 ) 2 ] Cb r (TX - b 0 ) + b 2 (TX - b 0 ) 2 
+ c x (TN - b 0 ) + c 2 (TN - b 0 ) 2 ] 


where 

DID = dally Increment of development 
DL = day length In hours 
TX = dally maximum air temperature 
TN = dally minimum air temperature 

a^,bj, and c^ are characteristic coefficients, 1 = 0, 1, 2 

The characteristic coefficients are specifically defined for each growth stage 
of the crop of Interest. Two attempts have been made to formulate 
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coefficients that would have more realistic physiological responses for spring 
wheat. The first was by Cate, et al. (ref. 5); the second Is from additional 
SR Project research. These agromet models for spring wheat are referred to In 
this document as the Robertson model; the Improved Robertson model, version 1; 
and the Improved Robertson model, version 2. Coefficients for a spring barley 
model have been developed by Williams. This basic form of crop development 
models requires an Initiation date. Starter models have been formulated to 
predict this Initiation date. Starter models may be Initiated simply by usl ig 
a historical average planting date or by using planting dates modified, for 
Instance, as a function of mean dally air temperature (Feyerherm, ref. 7). 

2.2 OBJECTIVES 

To satisfy the general objectives of this experiment, three Issues will be 
studied: 

1. Which combination of planting date model and growth stage model Is 
expected to most accurately predict the growth stage as a function of time 
for spring wheat and for barley? 

2. Does the Williams barley model more accurately predict the barley growth 
stages than the spring wheat model? 

3. Do the models selected perform well enough to be used In the labeling 
procedures? 

2.3 APPROACH 

This crop calendar evaluation naturally lends Itself to division Into four 
separate parts. The first two support Issue 1; the third and fourth support 
Issues 2 and 3, respectively. The four parts are: 

a. Planting date model evaluation 

b. Spring wheat growth stage model evaluation 

c. Williams barley model accuracy evaluation 

d. Biowindow prediction accuracy 
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These four parts of the evaluation will contribute substantially toward the 
selection of which spring wheat model to use, whether or not the barley model 
Is useful, and whether or not the Feyerherm starter model Is of value. 

The data set for the planting date model evaluation consists of ground truth 
observation of planting date for Individual spring wheat and barley fields, 
for historical normal planting dates for the area of the segment, and for 
Feyerherm planting date model predictions for the segment. 

The four crop growth stage models were started using (a) the observed field 
stages, (b) the median observed field stage, (c) the normal segment planting 
date, and (d) the Feyerherm segment planting date prediction. The growth 
stage model predictions were then made for each date of the periodic 
observation. 

The 1979 spring wheat and barley sites In the USNGP (standard data set 1R, see 
reference 8) contain about 51 segments with 9- or 18-day periodic observations 
of ground- truth growth stage development. Each segment has from 15 to 30 
spring wheat and barley fields observed. In total, -here are 6604 
observations for use In this study. 

2.4 PRODUCTS. REPORTS, AND DOCUMENTATION 

The technical aspects of this experiment will be monitored by a representative 
of the Experiment Design Component. Recommendations for future exploratory 
and pilot experiments will be provided for the PTRR and the final project 
experiment report. 


2-3 



3. ACCURACY ASSESSMENT 


3.1 OBJECTIVES 

The objectives of the Accuracy Assessment Component are to evaluate the 
accuracy of the planting date models and the crop calendar models and to 
determine the combination of planting date model and growth stage model which 
will most accurately predict the growth stage as a function of time for spring 
wheat and for barley. In addition, the berley crop calendar will be evaluated 
to determine If it predicts the barley growth stages more accurately than the 
spring wheat model. The final crop calendars selected will be eval»ated on 
their ability to produce results accurate enough to be used in the spring 
wheat and barley labeling procedure. 

*3.2 APPROACH 

The crop calendars will he evaluated using periodic observations from CY79 for 
51 segments In the spring wheat areas of the US6P. This data set consists of 
growth stage observations taken every 9 or 18 days for a number of spring 
wheat and barley fields In each segment and represents the most complete data 
set available. 

The planting date models and growth stage models will be evaluated Independ- 
ently to determine which planting date model is best and which growth stage 
model is best. These evaluations will be done on a regional basis to see if 
the models* performance varies from region to region. The final evaluation of 
the selected crop calendars will be done in terms of the crop calendars' 
ability to determine the acquisition windows defined by the reformatted spring 
wheat and barley labeling procedure. 

The evaluations will be performed on the Laboratory for Applications of Remote 
Sensing (LARS) computer system using the Statistical Analysis System (SAS) 
routines. The data for the evaluation will be provided by the Crop Calendar 
‘Component. This component will provide both the model predictions and peri- 
odic observations for each observation. The data will be provided in a disk 
file on the LARS computer system. 
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The following ranges of Robertson crop growth stages will be used to evaluate 
the performance of the various models for spring wheat and for barley: 

a. Crop stages 2.0 to 2.9 

b. Crop stages 3.0 to 3.9 

c. Crop stages 4.0 to 4.9 

d. Crop stages 5.0 to 5.9 

e. Crop stage 6.0 

In addition, the models will be evaluated for the entire growing season from 
stages 2.0 to 6.0 In order to estimate the overall performance of the models.' 

For stages 2.0 to 5.9, the Friedman Test (ref. 9) will be used to determine If 
one of the models produces significantly different results from the other 
models. The variable for this test will be the absolute value of the error 
produced by a model for a segment. Each segment will be a block. Each model 
will be a treatment. By ranking the model predictions for each segment, the 
Friedman Test can be used to determine If one model produces significantly 
better results than the other models. 

3.2.1 PLANTING DATE MODEL EVALUATION 

Two planting date models w*ll be evaluated. The first model provides the 
normal planting date for the segment based on historical data. The second 
model Is the Feyerherm model; this model uses weather data to determine the 
planting date. 

The models will be evaluated on their ability to accurately predict the 
observed median planting date for the segment. The accuracy will be charac- 
terized by the error In days between the model prediction and the observed 
date. Some measures of the accuracy which will be used to determine the best 
model are the mean error In days, the distribution of the error, and the mean 
square error (MSE). In addition, the percentage of the segments for which one 
model produces a better estimate than the other will be used to determine 
which model is best. 


3-2 



3.2.2 GROWTH STAGE MODEL EVALUATION FOR SPRING WHEAT 

As stated In section 2.1, three spring wheat growth stage models which will be 
evaluated: the original Robertson model and two Improved models. Each of the 
models uses weather data to determine the rate at which the growth stages 
change. 

The models will be evaluated using the observed median planting date as Input 
to the model. (In addition, the Individual field planting dates will be used 
to start the models to determine the ultimate performance of the models.) The 
evaluation will be made on the basis of the models' ability to accurately pre- 
dict the observed median growth stages. Evaluations will be completed for 
each of the stages In the observations. Some measures of the accuracy which 
will be used to determine the best model are the mean error In crop growth 
stage, the distribution of the error, and the percentage of the segments In 
which one model produced better estimates than the other. 

3.2.3 EFFECTIVENESS OF THE WILLIAMS BARLEY MODEL 

In order to determine If the barley growth stage model Is more accurate than 
the spring wheat model In predicting the barley growth stages, each model will 
be run using the observed median planting date for barley as Input. Using the 
criteria for growth stage model evaluation, the best barley growth stage 
predictor will be determined. If the spring wheat model Is better than the 
barley model In predicting the barley growth stage, then only the spring wheat 
model Is needed to predict the grovith stage for both spring wheat and barley. 

3.2.4 EVALUATION OF THE ACCURACY OF PREDICTING ACQUISITION WINDOW LOCATIONS 

The final choices for crop calendar models will be evaluated in terms of their 
ability to accurately predict the acquisition window locations as defined by 
the spring wheat/barley reformatted labeling procedure. The best spring wheat 
planting date and growth stage models will be used to predict the location of 
acquisition windows 1, 2, and 4. The best barley planting date and growth 
stage models will be used to predict the location of acquisition window 3. 

The predicted window location will be compared with the window location 


3-3 


derived from the observations. Two measures of accuracy which will be used 
are (a) the percent overlap between the predicted and observed window 
locations and (b) the displacement In days In the two-window locations. 

3.3 REPORTS AND DOCUMENTATION 

A "quick-look" report will be prepared containing the preliminary results and 
recommendations derived from this evaluation and a final accuracy assessment 
report will follow. Support will be provided for the PTRR and the final 
project report. 
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4. CROP CALENDARS 


4.1 OBJECTIVES 

The SR Project, has two primary objectives In the evaluation of agromet crop 
stage estimation models. The first is a statistical summary of model perform- 
ance sufficient to allow SR Project scientists to select the models for use in 
FCPF exploratory and pilot experiments. The second is an identification of 
any systematic model bias either by stage or geographical location. 

The Accuracy Assessment evaluation will provide the basis for model selection 
and identification of potential model bias. 

4.2 APPROACH 

The SR Project will provide to the Accuracy Assessment Component the following 
data products to be used in this evaluation: 

a. Normal planting dates for spring wheat and barley for each test segment 

b. Results of model runs along with coincident CY79 ground-truth observations 
of growth stage by field for spring wheat and barley 

Four models will be run: 

1. Robertson model 

2. Improved Robertson model, version 1 

3. Improved Robertson model, version 2 

4. Williams model 

These models will be started in the following ways: 

1. Feyerherm planting date estimate 

2. Segment median planting date based on ground truth for wheat and barley 

3. Actual planting date for each wheat and barley field 

4. Normal planting date for each segment 

c. The actual planting date for each field along with the Feyerherm planting 

date estimate 
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All products win be delivered to the Accuracy Assessment Component In the 
form of data files on the LARS computer system. The format of the data files 
will be coordinated with Accuracy Assessment. 

All models In this study will be run dally meteorological data taken at the 
nearest possible climatological station to the test segments. 

In addition to data products, the SR Project will provide technical advice as 
needed. 

4 . 3 PRODUCTS, REPORTS. AND DOCUMENTATION 

The SR project will provide to the Accuracy Assessment Component the required 
(a) normal planting dates, ( b) results of model runs with coincident ground- 
truth observations, and (c) Feyerherm-model -determined planting dates and 
actual planting dates in the form of data files as outlined In section 4.2. 

Support for the preparation of the PTRR will be provided. Recommendations 
based upon the Accuracy Assessment evaluation regarding model selection and 
further crop calendar research will be submitted for Inclusion In the final 
U.S. /Canada Wheat and Barley Exploratory Experiment report. 



5. DATA AND SYSTEMS REQUIREMENTS SUMMARY 


The data and systems requirements necessary to support the components of this 
experiment are summarized in this section. 

5.1 EXPERIMENT DESIGN COMPONENT 

In order to make recommendations for future exploratory and pilot experiments, 
the Experiment Design Component requires the final accuracy assessment 
analysis results. 

5.2 ACCURACY ASSESSMENT COMPONENT 

The Accuracy Assessment Component will receive the model of predicted and 
observed data from the Crop Calendar Component in the form of a disk file on 
the LARS computer system. The SAS routines on the LARS computer system will 
be used to perform the evaluations. 

5.3 CROP CALENDAR COMPONENT 

Meteorological data for model runs will be extracted from literature available 
locally. Ground observations are currently available within the Crop Calendar 
Component. Model runs will be made on the LARS 3031 computer. 
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6. EXPERIMENT SCHEDULE AND RESOURCE SUMMARY 


Pursuant to the FCPF Implementation Plan (ref. 1), an exploratory experiment 
for wheat and barley In the United States and Canada has been scheduled for 
FY80. Supplemental to this experiment, a small grain wheat and barley explor- 
atory proportion estimation experiment and a crop calendar experiment have 
been planned. The testing and evaluation of candidate procedures and crop 
calendars for follow-on exploratory and pilot experiments will be reported in 
a PTRR followed by a formal document early in FY81. The PTRR presentation is 
currently scheduled (consistent with data and critical resource availability) 
to allow the earliest possible incorporation of findings into FY81 experiment, 
pi anning. 

6.1 EXPERIMENT SCHEDULES 

In order for an experiment to arrive at a successful conclusion, an integrated 
schedule must be developed portraying each component's relationship to the 
other. The experiment schedule shows this relationship in summary form 
(fig. 6-1). 

6.2 RESOURCE SUMMARY 

Resources necessary to conduct this exploratory experiment will be drawn from 
those provided for the project as defined by task sheet in the FCPF and SR 
Projects Implementation Plans. Resource scoping across organizational 
elements is generally consistent with individual task sheet estimates. Both 
civil service and contractor personnel are involved to varying degrees in all 
facets of the experiment. The following sections present the projected 
requirements for civil service and contractor personnel. 

6.2.1 CIVIL SERVICE RESOURCE SUMMARY 

Civil service personnel are engaged in numerous tasks relating to Project 
Management and Support that are involved in directing and monitoring the 
Experiment Design, Crop Calendar, and Accuracy Assessment contractor effort. 
Total civil service manpower involved in this experiment is 2 man-weeks. 
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6.2.2 CONTRACTOR RESOURCE SUMMARY 


Contractor personnel participate in the implementation of this experiment 
through all of the components. The level of manpower involvement on a per 
component basis is shown in table 6-1. 

TABLE 6-1.- CONTRACTOR MAN-YEAR EQUIVALENTS (MYE) PER COMPONENT 


Section 

Component 

Support Contractor MYE 

1 . 

Experiment Plan Development 

0.1 

2. 

Experiment Design 

0.1 

3. 

Accuracy Assessment 

0.25 

4. 

Crop Calendars 

0.1 

5. 

Data and Systems Summary 

— 

6. 

Experiment and Resource Summary 

— 


6-3 









7. REFERENCES 


1. AgRISTARS — Foreign Commodity Production Forecasting Implementation Plan. 
NASA-JSC, JSC-16344, January 15, 1980. 

2. AgRISTARS/FCPF - U.S. /Canada Wheat and Barley Exploratory Labeling 
Experiment Implementation Plan. NASA-JSC, JSC-16336, January 1980. 

3. AgRISTARS/FCPF - Supplemental U.S. /Canada Wheat and Barley Exploratory 
Experiment Implementation Plan: Evaluation of a Procedure 1A Technology. 
NASA-JSC, JSC-16364, June 1980. 

4. Woolley, S. K.; Whitehead, V. S.; Stuff, R. G.; and Crea, W. E.: Accuracy 
and Performance of LAC IE Crop Development Models. Proceedings of Techni- 
cal Sessions — The LACIE Symposium (October 1978) NASA-JSC, JSC-16015, 

July 1979. 

5. Cate, R. B.; Artley, J. A.; Doraiswamy, P. C.; Hodges, T.; Kinsler, M. C,«; 

Phinney, D. E.; and Sestak, M. L.: Preliminary Evaluation of Spectral, 

Normal, and Meteorological Crop Stage Estimation Approaches. NASA-JSC, 
LEMSC0-14640 (to be published). 

6. Hodges, T.; and Doraiswamy, P. C.: Crop Phenology Literature Review for 
Corn, Soybean, Wheat, Barley, Sorghum, Rice, Cotton, and Sunflower. 
NASA-JSC, JSC-16088, LEC-13722, November 1979. 

7. Feyerherm, A. M.: Planting Date and Wheat Yield Models. Final Report for 

the Period February 1975 to March 1977. NASA-JSC (Contract NAS 9-14533) 
Kansas State University, September 1977. 

8. AgRISTARS — Supporting Research Implementation Plan, Appendix D. 

NASA-JSC, September 14, 1979. 

9. Conover, W. J.: Practical Nonparametric Statistics. John Wiley and Sons, 
Inc., 1971. 


U, S. GOVERNMENT PRINTING OFFICE: 1981-771-014/196 


